I guess this caused performance regression for colours? Or maybe it is just a fluctuation, we will see now. In general I think this may cause performance regression for code with many subtype checks with negative outcome, most notably in overloads checking (which is likely why colours is so slow in the first place).

Let's see if the restart helps - I've seen this happening randomly elsewhere.

Isolated benchmarks also look good: sterliakov/mypy-issues#110

This can have negative effect when almost all subtype checks are expected to fail, but the check should be fast enough to compensate for that. Looking at primer logs, all other projects I looked at (~20 instances from five shards) were checked faster with this patch, so this might be net positive anyway.

Some long-running examples:

/tmp/mypy_primer/mypy_new/venv/bin/mypy on core took 583.57s
/tmp/mypy_primer/mypy_old/venv/bin/mypy on core took 608.09s

/tmp/mypy_primer/mypy_new/venv/bin/mypy on rclip took 30.12s
/tmp/mypy_primer/mypy_old/venv/bin/mypy on rclip took 31.93s

/tmp/mypy_primer/mypy_new/venv/bin/mypy on svcs took 61.66s
/tmp/mypy_primer/mypy_old/venv/bin/mypy on svcs took 63.76s

/tmp/mypy_primer/mypy_new/venv/bin/mypy on arviz took 145.72s
/tmp/mypy_primer/mypy_old/venv/bin/mypy on arviz took 148.53s

This is obviously not the best indicator, but at least something...

colour, however, seems to demonstrate negative impact:

/tmp/mypy_primer/mypy_old/venv/bin/mypy on colour took 3223.60s
/tmp/mypy_primer/mypy_new/venv/bin/mypy on colour took 3298.71s

It's not clear whether any of those are reliable, because the run for #19388 shows significantly lower time with expected positive improvement, but is 3 minutes less for master branch:

/tmp/mypy_primer/mypy_new/venv/bin/mypy on colour took 3013.21s
/tmp/mypy_primer/mypy_old/venv/bin/mypy on colour took 3095.46s

(failing join_artifacts has nothing to do with this PR, it's likely a github outage: https://github.com/python/mypy/actions/runs/16144405699/job/45564336755?pr=19400)

And let's also note that primer runs interpreted mypy, and this equality should play well with mypyc. I'll try running perf_compare on colour today.

In this run there's no colour regression:

/tmp/mypy_primer/mypy_new/venv/bin/mypy on colour took 2875.19s
/tmp/mypy_primer/mypy_old/venv/bin/mypy on colour took 2939.26s

but "need type annotation" is mysterious.

Okay, nothing mysterious, just a separate bug. Please ping me if you don't think this PR should ever be merged, I will extract the bugfix to a separate PR - def fn(x) -> bool and def fn(x) -> TypeGuard[int] definitely should not compare equal.

This was fun to debug - I ended up with assert sorted(repr(left)) == sorted(repr(right)) in these early return branches to check that equality works as expected, and got a jaw-dropping moment when it actually failed.

And this does indeed improve colour check time, the first run was apparently a glitch or artifact running checks concurrently, it became faster in its own shard. Compiled benchmark (with numpy and other relevant packages installed) shows -2.4% over 3 runs (each run takes ~4 min on my PC so I wasn't ready to wait longer).

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅